Example for cross sectional flux method¶

Author: Gerrit Kuhlmann (gerrit.kuhlmann@empa.ch)

This example code applies the cross sectional flux method to TROPOMI NO2 observations to quantify the emissions of the Janschwalde power plant.

In [1]:
%matplotlib inline

import os
import numpy as np
import pandas as pd
import scipy.stats
import xarray as xr

import ddeq
import ucat

Create source dataset and setup domain for plotting¶

In [2]:
# Source dataset
sources = xr.Dataset({
    "source": xr.DataArray(["6298"], dims="source"),
    "label": xr.DataArray(["Janschwalde"], dims="source"),
    "lon": xr.DataArray([14.457959], dims="source"),
    "lat": xr.DataArray([51.836241], dims="source"),
    "diameter": xr.DataArray([1000.0], dims="source"),
    "NOx_emissions": xr.DataArray([0.42778918548939077], dims="source"), # Bottom-up estimate of emissions (in kg/s)
})

# Setup domain for plotting
NAME = "6298" # CORSO ID for Janschwalde power plant
SOURCE = sources.sel(source=[NAME])
DOMAIN = ddeq.misc.Domain.around_source(SOURCE)
CRS = ddeq.misc.get_opt_crs(DOMAIN)

# CSF settings going 30 km downstream and 80 km across
DMAX = 30e3
PLUME_WIDTH = 80e3

Read TROPOMI Level-2 data¶

The following code reads the TROPOMI Level-2 file (S5P_RPRO_L2__NO2____20210308T110658_20210308T124828_17622_03_020400_20221107T150153.nc"). You can download TROPOMI data following the example_download_tropomi.ipynb notebook in the ddeq library.

The ddeq.sats.read_S5P function will collect the different variables from the various groups to create a xarray.Dataset ready for ddeq:

In [3]:
filename = "/scratch/koer/TROPOMI/NO2/S5P_RPRO_L2__NO2____20210308T110658_20210308T124828_17622_03_020400_20221107T150153.nc"
data = ddeq.sats.read_S5P(filename, gas="NO2")
data
Out[3]:
<xarray.Dataset> Size: 399MB
Dimensions:                      (scanline: 4173, ground_pixel: 450, corner: 4,
                                  layer: 34, vertices: 2)
Coordinates:
  * scanline                     (scanline) float64 33kB 0.0 1.0 ... 4.172e+03
  * ground_pixel                 (ground_pixel) float64 4kB 0.0 1.0 ... 449.0
  * corner                       (corner) float64 32B 0.0 1.0 2.0 3.0
  * layer                        (layer) float64 272B 0.0 1.0 2.0 ... 32.0 33.0
  * vertices                     (vertices) float64 16B 0.0 1.0
    lat                          (scanline, ground_pixel) float32 8MB -73.55 ...
    lon                          (scanline, ground_pixel) float32 8MB -134.6 ...
    orbit                        int64 8B 17622
Data variables: (12/16)
    time                         (scanline) datetime64[ns] 33kB 2021-03-08T11...
    time_utc                     (scanline) <U27 451kB '2021-03-08T11:28:32.2...
    qa_value                     (scanline, ground_pixel) float32 8MB 0.0 ......
    NO2                          (scanline, ground_pixel) float32 8MB nan ......
    NO2_std                      (scanline, ground_pixel) float32 8MB nan ......
    averaging_kernel             (scanline, ground_pixel, layer) float32 255MB ...
    ...                           ...
    solar_zenith_angle           (scanline, ground_pixel) float32 8MB ...
    latitude_bounds              (scanline, ground_pixel, corner) float32 30MB ...
    longitude_bounds             (scanline, ground_pixel, corner) float32 30MB ...
    surface_altitude             (scanline, ground_pixel) float32 8MB ...
    surface_pressure             (scanline, ground_pixel) float32 8MB ...
    cloud_fraction_crb           (scanline, ground_pixel) float32 8MB ...
Attributes:
    original file name:  S5P_RPRO_L2__NO2____20210308T110658_20210308T124828_...
    data source:         https://s5phub.copernicus.eu/dhus/
    time:                2021-03-08T11:57:44.461487658
xarray.Dataset
    • scanline: 4173
    • ground_pixel: 450
    • corner: 4
    • layer: 34
    • vertices: 2
    • scanline
      (scanline)
      float64
      0.0 1.0 2.0 ... 4.171e+03 4.172e+03
      units :
      1
      axis :
      Y
      long_name :
      along-track dimension index
      comment :
      This coordinate variable defines the indices along track; index starts at 0
      array([0.000e+00, 1.000e+00, 2.000e+00, ..., 4.170e+03, 4.171e+03, 4.172e+03])
    • ground_pixel
      (ground_pixel)
      float64
      0.0 1.0 2.0 ... 447.0 448.0 449.0
      units :
      1
      axis :
      X
      long_name :
      across-track dimension index
      comment :
      This coordinate variable defines the indices across track, from west to east; index starts at 0
      array([  0.,   1.,   2., ..., 447., 448., 449.])
    • corner
      (corner)
      float64
      0.0 1.0 2.0 3.0
      units :
      1
      long_name :
      pixel corner index
      comment :
      This coordinate variable defines the indices for the pixel corners; index starts at 0 (counter-clockwise, starting from south-western corner of the pixel in ascending part of the orbit)
      array([0., 1., 2., 3.])
    • layer
      (layer)
      float64
      0.0 1.0 2.0 3.0 ... 31.0 32.0 33.0
      standard_name :
      atmosphere_hybrid_sigma_pressure_coordinate
      units :
      1
      long_name :
      TM5 atmospheric layer numbers
      positive :
      down
      axis :
      Z
      formula_terms :
      ap: tm5_constant_a b: tm5_constant_b ps: /PRODUCT/SUPPORT_DATA/INPUT_DATA/surface_pressure
      comment :
      p(t, k, j, i, l) = ap(k, l) + b(k, l)*ps(t, j, i); k from surface to top of atmosphere; l=0 for base of layer, l=1 for top of layer.
      array([ 0.,  1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  9., 10., 11., 12., 13.,
             14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27.,
             28., 29., 30., 31., 32., 33.])
    • vertices
      (vertices)
      float64
      0.0 1.0
      units :
      1
      long_name :
      TM5 atmospheric layer upper and lower bound indices
      array([0., 1.])
    • lat
      (scanline, ground_pixel)
      float32
      -73.55 -73.58 ... 79.25 79.24
      long_name :
      pixel center latitude
      units :
      degrees_north
      standard_name :
      latitude
      valid_min :
      -90.0
      valid_max :
      90.0
      bounds :
      /PRODUCT/SUPPORT_DATA/GEOLOCATIONS/latitude_bounds
      array([[-73.5502  , -73.57943 , -73.607834, ..., -63.346405, -63.27331 ,
              -63.19915 ],
             [-73.59649 , -73.625786, -73.65426 , ..., -63.377148, -63.303993,
              -63.22977 ],
             [-73.64275 , -73.67212 , -73.70066 , ..., -63.407764, -63.33455 ,
              -63.260265],
             ...,
             [ 66.49975 ,  66.57008 ,  66.63935 , ...,  79.35066 ,  79.33973 ,
               79.32804 ],
             [ 66.477295,  66.54753 ,  66.61671 , ...,  79.30502 ,  79.29421 ,
               79.28265 ],
             [ 66.45483 ,  66.52497 ,  66.594055, ...,  79.25933 ,  79.24864 ,
               79.237206]], dtype=float32)
    • lon
      (scanline, ground_pixel)
      float32
      -134.6 -134.9 ... 176.8 176.4
      long_name :
      pixel center longitude
      units :
      degrees_east
      standard_name :
      longitude
      valid_min :
      -180.0
      valid_max :
      180.0
      bounds :
      /PRODUCT/SUPPORT_DATA/GEOLOCATIONS/longitude_bounds
      array([[-134.57545 , -134.87038 , -135.16214 , ...,  158.94234 ,  158.84938 ,
               158.75562 ],
             [-134.54324 , -134.8389  , -135.1314  , ...,  158.85396 ,  158.76114 ,
               158.66751 ],
             [-134.511   , -134.80737 , -135.1006  , ...,  158.76535 ,  158.67265 ,
               158.57916 ],
             ...,
             [-101.17287 , -101.29536 , -101.41683 , ...,  177.107   ,  176.67131 ,
               176.23067 ],
             [-101.28589 , -101.4086  , -101.53029 , ...,  177.178   ,  176.74405 ,
               176.30516 ],
             [-101.398735, -101.52167 , -101.64357 , ...,  177.24794 ,  176.81573 ,
               176.37859 ]], dtype=float32)
    • orbit
      ()
      int64
      17622
      array(17622)
    • time
      (scanline)
      datetime64[ns]
      2021-03-08T11:28:32.261000 ... 2...
      long_name :
      Time of observation
      array(['2021-03-08T11:28:32.261000000', '2021-03-08T11:28:33.101000000',
             '2021-03-08T11:28:33.941000000', ..., '2021-03-08T12:26:54.981000000',
             '2021-03-08T12:26:55.821000000', '2021-03-08T12:26:56.661000000'],
            dtype='datetime64[ns]')
    • time_utc
      (scanline)
      <U27
      '2021-03-08T11:28:32.261000Z' .....
      long_name :
      Time of observation as ISO 8601 date-time string
      array(['2021-03-08T11:28:32.261000Z', '2021-03-08T11:28:33.101000Z',
             '2021-03-08T11:28:33.941000Z', ..., '2021-03-08T12:26:54.981000Z',
             '2021-03-08T12:26:55.821000Z', '2021-03-08T12:26:56.661000Z'],
            dtype='<U27')
    • qa_value
      (scanline, ground_pixel)
      float32
      0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 0.0
      units :
      1
      valid_min :
      0
      valid_max :
      100
      long_name :
      data quality value
      comment :
      A continuous quality descriptor, varying between 0 (no data) and 1 (full quality data). Recommend to ignore data with qa_value < 0.5
      array([[0., 0., 0., ..., 0., 0., 0.],
             [0., 0., 0., ..., 0., 0., 0.],
             [0., 0., 0., ..., 0., 0., 0.],
             ...,
             [0., 0., 0., ..., 0., 0., 0.],
             [0., 0., 0., ..., 0., 0., 0.],
             [0., 0., 0., ..., 0., 0., 0.]], dtype=float32)
    • NO2
      (scanline, ground_pixel)
      float32
      nan nan nan nan ... nan nan nan nan
      units :
      mol m-2
      standard_name :
      troposphere_mole_content_of_nitrogen_dioxide
      long_name :
      Tropospheric vertical column of nitrogen dioxide
      ancillary_variables :
      nitrogendioxide_tropospheric_column_precision air_mass_factor_troposphere air_mass_factor_total averaging_kernel
      multiplication_factor_to_convert_to_molecules_percm2 :
      6.02214e+19
      noise_level :
      1.1183850801899098e-05
      array([[nan, nan, nan, ..., nan, nan, nan],
             [nan, nan, nan, ..., nan, nan, nan],
             [nan, nan, nan, ..., nan, nan, nan],
             ...,
             [nan, nan, nan, ..., nan, nan, nan],
             [nan, nan, nan, ..., nan, nan, nan],
             [nan, nan, nan, ..., nan, nan, nan]], dtype=float32)
    • NO2_std
      (scanline, ground_pixel)
      float32
      nan nan nan nan ... nan nan nan nan
      units :
      mol m-2
      standard_name :
      troposphere_mole_content_of_nitrogen_dioxide standard_error
      long_name :
      Precision of the tropospheric vertical column of nitrogen dioxide
      multiplication_factor_to_convert_to_molecules_percm2 :
      6.022141e+19
      array([[nan, nan, nan, ..., nan, nan, nan],
             [nan, nan, nan, ..., nan, nan, nan],
             [nan, nan, nan, ..., nan, nan, nan],
             ...,
             [nan, nan, nan, ..., nan, nan, nan],
             [nan, nan, nan, ..., nan, nan, nan],
             [nan, nan, nan, ..., nan, nan, nan]], dtype=float32)
    • averaging_kernel
      (scanline, ground_pixel, layer)
      float32
      ...
      units :
      1
      long_name :
      Averaging kernel
      ancillary_variables :
      tm5_constant_a tm5_constant_b tm5_tropopause_layer_index /PRODUCT/SUPPORT_DATA/INPUT_DATA/surface_pressure
      [63846900 values with dtype=float32]
    • air_mass_factor_troposphere
      (scanline, ground_pixel)
      float32
      ...
      units :
      1
      long_name :
      Tropospheric air mass factor
      ancillary_variables :
      tm5_tropopause_layer_index
      [1877850 values with dtype=float32]
    • air_mass_factor_total
      (scanline, ground_pixel)
      float32
      ...
      units :
      1
      long_name :
      Total air mass factor
      [1877850 values with dtype=float32]
    • tm5_constant_a
      (layer, vertices)
      float32
      ...
      units :
      Pa
      long_name :
      TM5 hybrid A coefficient at upper and lower interface levels
      [68 values with dtype=float32]
    • tm5_constant_b
      (layer, vertices)
      float32
      ...
      units :
      1
      long_name :
      TM5 hybrid B coefficient at upper and lower interface levels
      [68 values with dtype=float32]
    • solar_zenith_angle
      (scanline, ground_pixel)
      float32
      ...
      long_name :
      solar zenith angle
      standard_name :
      solar_zenith_angle
      units :
      degree
      valid_min :
      0.0
      valid_max :
      180.0
      comment :
      Solar zenith angle at the ground pixel location on the reference ellipsoid. Angle is measured away from the vertical
      [1877850 values with dtype=float32]
    • latitude_bounds
      (scanline, ground_pixel, corner)
      float32
      ...
      [7511400 values with dtype=float32]
    • longitude_bounds
      (scanline, ground_pixel, corner)
      float32
      ...
      [7511400 values with dtype=float32]
    • surface_altitude
      (scanline, ground_pixel)
      float32
      ...
      long_name :
      Surface altitude
      standard_name :
      surface_altitude
      units :
      m
      source :
      http://topotools.cr.usgs.gov/gmted_viewer/
      comment :
      The mean of the sub-pixels of the surface altitudewithin the approximate field of view, based on the GMTED2010 surface elevation database
      [1877850 values with dtype=float32]
    • surface_pressure
      (scanline, ground_pixel)
      float32
      ...
      units :
      Pa
      standard_name :
      surface_air_pressure
      long_name :
      Surface pressure
      [1877850 values with dtype=float32]
    • cloud_fraction_crb
      (scanline, ground_pixel)
      float32
      ...
      units :
      1
      proposed_standard_name :
      effective_cloud_area_fraction_assuming_fixed_cloud_albedo
      long_name :
      Effective cloud fraction from the cloud product
      ancillary_variables :
      /PRODUCT/SUPPORT_DATA/DETAILED_RESULTS/cloud_selection_flag
      [1877850 values with dtype=float32]
    • scanline
      PandasIndex
      PandasIndex(Index([   0.0,    1.0,    2.0,    3.0,    4.0,    5.0,    6.0,    7.0,    8.0,
                9.0,
             ...
             4163.0, 4164.0, 4165.0, 4166.0, 4167.0, 4168.0, 4169.0, 4170.0, 4171.0,
             4172.0],
            dtype='float64', name='scanline', length=4173))
    • ground_pixel
      PandasIndex
      PandasIndex(Index([  0.0,   1.0,   2.0,   3.0,   4.0,   5.0,   6.0,   7.0,   8.0,   9.0,
             ...
             440.0, 441.0, 442.0, 443.0, 444.0, 445.0, 446.0, 447.0, 448.0, 449.0],
            dtype='float64', name='ground_pixel', length=450))
    • corner
      PandasIndex
      PandasIndex(Index([0.0, 1.0, 2.0, 3.0], dtype='float64', name='corner'))
    • layer
      PandasIndex
      PandasIndex(Index([ 0.0,  1.0,  2.0,  3.0,  4.0,  5.0,  6.0,  7.0,  8.0,  9.0, 10.0, 11.0,
             12.0, 13.0, 14.0, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0,
             24.0, 25.0, 26.0, 27.0, 28.0, 29.0, 30.0, 31.0, 32.0, 33.0],
            dtype='float64', name='layer'))
    • vertices
      PandasIndex
      PandasIndex(Index([0.0, 1.0], dtype='float64', name='vertices'))
  • original file name :
    S5P_RPRO_L2__NO2____20210308T110658_20210308T124828_17622_03_020400_20221107T150153.nc
    data source :
    https://s5phub.copernicus.eu/dhus/
    time :
    2021-03-08T11:57:44.461487658

Iterate over Level-2 orbit to find Janschwalde¶

The following code will iterate over small chunks of the Level-2 orbit to find the Janschwalde power plant in the swath. The plume area will be identified using the ERA-5 wind vector.

This requires ERA-5 data on pressure levels and on single level, which can be downloade following the example_tropomi_Matimba.ipynb and example_effective_winds.ipynb notebook in the ddeq library.

In [4]:
for chunk in ddeq.sats.iter_S5P_swath(filename, gas="NO2", preprocessed=False):

    # Plume detection using wind vector
    time = pd.Timestamp(chunk.attrs['time'])
    lvl_filename = time.strftime('/input/ERA5/CORSO/raw/ERA5-lvl-%Y%m%dt0000.nc')
    sng_filename = time.strftime('/input/ERA5/CORSO/raw/ERA5-sng-%Y%m%dt0000.nc')

    winds = ddeq.era5.read(
        sng_filename,
        lvl_filename,
        method='gnfra',
        times=time,
        extent=DOMAIN.extent,
        sources=SOURCE
    )

    chunk = ddeq.dplume.detect_from_wind(chunk, SOURCE, winds, dmax=DMAX, crs=CRS)

    if "x_nodes" in chunk: # center line was computed, i.e. Janschwalde is inside chunk
        break

# Prepare local coordinate system, plume area and convert from µmol/m² to kg/m²
data = ddeq.curves.compute_natural_coords(chunk)
data = ddeq.curves.compute_plume_areas(data, plume_width=PLUME_WIDTH)
ddeq.emissions.convert_units(data, "NO2", "NO2")

Quantify emissions using NO2 observations¶

The first estimate of NOx emissions is obtained by applying the CSF method with standard values for NOx correction:

In [5]:
# Estimate emissions using CSF method (first estimate without any corrections)
emissions = ddeq.csf.estimate_emissions(
    data,
    winds,
    SOURCE,
    'NO2',
    xmin=1e3,
    f_model=1.32,             # NO2-to-NOx conversion factor
    decay_time=4.0 * 60**2,   # NOX lifetime (4 hours)
    crs=CRS,
    variable="NO2_mass",
    background="linear"
)

# Visualize the result
fig = ddeq.vis.plot_csf_result_compact(data, winds, emissions, SOURCE, NAME, sources, crs=CRS, domain=DOMAIN)
No description has been provided for this image

Air mass correction¶

AMF correction uses the NO$_2$ enhancement from the first Gaussian curve to update the NO$_2$ profiles from the TROPOMI AUX dataset. In this example, ddeq looks for the file "S5P_OPER_AUX_CTMANA_20210308T000000_20210309T000000_20221231T072039.nc" in the provided path.

In [6]:
# Air mass factor correction using enhancements from first Gaussian plume
data = ddeq.amf.correct_amf_from_csf_method(
    data,
    emissions,
    source_name=NAME,
    path="/input/CORSO/TROPOMI/aux/"
)

# 2nd emission quantification after AMF correction
emissions = ddeq.csf.estimate_emissions(
    data,
    winds,
    SOURCE,
    'NO2',
    xmin=1e3,
    f_model=1.32,
    decay_time=4.0 * 60**2,
    crs=CRS,
    variable="NO2_amf_mass",
    background="linear"
)

fig = ddeq.vis.plot_csf_result_compact(data, winds, emissions, SOURCE, NAME, sources, crs=CRS, domain=DOMAIN)
No description has been provided for this image

NOx correction and uncertainty estimation¶

The following code applies the local NOx correction using the empirical formula from Meier et al. 2024. The code also conducts a Monte Carlo simulation to estimate the uncertainty of the estimated emissions.

In [7]:
# Prepare emissions dataset (as code below assumes a time series)
emissions = xr.concat([emissions.copy()], dim="time")
N = emissions.time.size

# Line density ensemble
q = np.squeeze(emissions.NO2_line_density.values)
q_std = np.squeeze(emissions.NO2_line_density_precision.values)
q_sample = q + q_std * np.random.randn(10000)

# Wind speed ensemble using Log-normal distribution with mean of u_eff and RMSE of about 1.0 m/s
u = emissions.wind_speed.values.flatten()
u_std = 1.0
mu = np.log(u**2 / np.sqrt(u**2 + u_std**2))
sigma = np.sqrt(np.log(1 + u_std**2 / u**2))
u_sample = np.exp(mu + sigma * np.random.randn(10000, N))
In [8]:
# Compute NO2-to-NOx conversion (between 1 km and 30 km downstream of source
along = np.linspace(1, 30, 200) * 1e3

# NO2-to-NOx conversion ensemble
seconds_since_emission = along[None,:] / u[:,None]
seconds_sample = along[None,:,None] / u_sample[:,None,:]

NO2toNOx_model = ddeq.functions.NO2toNOxConversion(name="Janschwalde")

f_local, f_local_std = NO2toNOx_model(seconds_since_emission)
f_local_sample, _ = NO2toNOx_model(seconds_sample)
f_local_sample = f_local_sample.mean(1) + f_local_std.mean(1)[None,:] * np.random.randn(10000,N)

# NOx lifetime ensemble with median of 2.5 hours
tau = 2.5 * 60**2     # in seconds
tau_std = 0.8 * 60**2 # in seconds
mu = np.log(tau**2 / np.sqrt(tau**2 + tau_std**2))
sigma = np.sqrt(np.log(1 + tau_std**2 / tau**2))
tau_local_sample = np.exp(mu + sigma * np.random.randn(10000, N))

# Decay correction term
D_local_sample = np.exp(-along[None,:,None] / (tau_local_sample[:,None,:] * u_sample[:,None,:])).mean(1)

# NOx correction term
c_local_sample = f_local_sample / D_local_sample

# Create xarray dataset
sample = xr.Dataset()
sample["time"] = xr.DataArray(emissions.time.values, dims="time")
sample["model"] = xr.DataArray(["local"], dims="model")

sample["wind_speed"] = xr.DataArray(u_sample, dims=("n", "time"))

dims = ("model", "n", "time")
sample["NOx_NO2_ratio"] = xr.DataArray([f_local_sample], dims=dims, attrs={"models": ("Janschwalde",)})
sample["NOx_lifetime"] = xr.DataArray([tau_local_sample], dims=dims, attrs={"units": "s"})
sample["NOx_decay_correction"] = xr.DataArray([D_local_sample], dims=dims)
sample["NOx_correction"] = xr.DataArray([c_local_sample], dims=dims)
In [9]:
# Compute NOx emissions from NOx correction (c), line density (q) and wind speed (u)
u_sample = sample["wind_speed"].values
c_sample = sample["NOx_correction"].sel(model="local").values
Q_sample = c_sample * q_sample * u_sample

# Median and +/- sigma uncertainty
Q_low, Q_mid, Q_high = scipy.stats.scoreatpercentile(Q_sample, [50-34, 50, 50+34] )
f_low, f_mid, f_high = scipy.stats.scoreatpercentile(sample["NOx_NO2_ratio"], [50-34, 50, 50+34] )

# Apply update to results dataset
emissions.f[:] = f_mid
emissions.f_precision[:] = (f_high - f_low) / 2
emissions.NOx_decay_time[:] = 2.5 * 60**2

if np.squeeze(emissions.NO2_shift) > 10e3 or np.squeeze(emissions.NO2_standard_width) > 10e3:
    emissions.NOx_emissions[:] = np.nan
    emissions.NOx_emissions_precision[:] = np.nan
else:
    emissions.NOx_emissions[:] = Q_mid
    emissions.NOx_emissions_precision[:] = (Q_high - Q_low) / 2

# Plot final result
fig = ddeq.vis.plot_csf_result_compact(data, winds, emissions.isel(time=0), SOURCE, NAME, sources, crs=CRS, domain=DOMAIN)
No description has been provided for this image
In [ ]: